Layered control architectures in natural and artificial systems

We review recent research in robotics and neuroscience with the aim of highlighting some points of agreement and convergence. Specifically, we compare Brooks’ [9] subsumption architecture for robot control with a part of the neuroscience literature that can be interpreted as demonstrating hierarchical control systems in animal brains. We focus first on work that follows the tradition of Hughlings Jackson [23] who, in neuroscience and neuropsychology, is particularly associated with the notion of layered competence. From this perspective we further argue that recent work on the defense system of the rat can be interpreted by analogy to Brooks’ subsumption architecture. An important focus is the role of multiple learning systems in the brain, and of hierarchical learning mechanisms in the rat defense system

Layered control architectures in natural and artificial systems

We review recent research in robotics and neuroscience with the aim of highlighting some points of agreement and convergence. Specifically, we compare Brooks’ [9] subsumption architecture for robot control with a part of the neuroscience literature that can be interpreted as demonstrating hierarchical control systems in animal brains. We focus first on work that follows the tradition of Hughlings Jackson [23] who, in neuroscience and neuropsychology, is particularly associated with the notion of layered competence. From this perspective we further argue that recent work on the defense system of the rat can be interpreted by analogy to Brooks’ subsumption architecture. An important focus is the role of multiple learning systems in the brain, and of hierarchical learning mechanisms in the rat defense system

The basal ganglia: A vertebrate solution to the selection problem?

A selection problem arises whenever two or more competing systems seek simultaneous access to a restricted resource. Consideration of several selection architectures suggests there are significant advantages for systems which incorporate a central switching mechanism. We propose that the vertebrate basal ganglia have evolved as a centralized selection device, specialized to resolve conflicts over access to limited motor and cognitive resources. Analysis of basal ganglia functional architecture and its position within a wider anatomical framework suggests it can satisfy many of the requirements expected of an efficient selection mechanism

Is the short-latency dopamine response too short to signal reward error?

Unexpected stimuli that are behaviourally significant have the capacity to elicit a short-latency, short-duration burst of firing in mesencephalic dopaminergic neurones. An influential interpretation of the experimental data that characterize this response proposes that dopaminergic neurones have a crucial role in reinforcement learning because they signal error in the prediction of future reward. In this article we propose a different functional role for this ‘short-latency dopamine response’ in the mechanisms that underlie associative learning. We suggest that the initial burst of dopaminergic-neurone firing could represent an essential component in the process of switching attentional and behavioural selections to unexpected, behaviourally important stimuli. This switching response could be a crucial prerequisite for associative learning and might be part of a general short-latency response that is mediated by catecholamines and prepares the organism for an appropriate reaction to biologically significant events. Any act which in a given situation produces satisfaction becomes associated with that situation so that when the situation recurs the act is more likely than before to recur also. E.L. Thorndike (1911)

The basal ganglia viewed as an action selection device

The action selection problem describes the task of resolving conflicts between the different functional systems that can control behavior. This paper reviews the role of the basal ganglia (BG) summarising evidence that they function within the vertebrate brain architecture as a specialized action selection device. There is a rich connectivity within the BG whose function is not well understood. We outline a new computational model of BG intrinsic pathways which demonstrates that these circuits could allow the BG to implement clean switching between competing functional systems

Empirically inspired simulated electro-mechanical model of the rat mystacial follicle-sinus complex

In whiskered animals, activity is evoked in the primary sensory afferent cells (trigeminal nerve) by mechanical stimulation of the whiskers. In some cell populations this activity is correlated well with continuous stimulus parameters such as whisker deflection magnitude, but in others it is observed to represent events such as whisker-stimulator contact or detachment. The transduction process is mediated by the mechanics of the whisker shaft and follicle-sinus complex (FSC), and the mechanics and electro-chemistry of mechanoreceptors within the FSC. An understanding of this transduction process and the nature of the primary neural codes generated is crucial for understanding more central sensory processing in the thalamus and cortex. However, the details of the peripheral processing are currently poorly understood. To overcome this deficiency in our knowledge, we constructed a simulated electro-mechanical model of the whisker-FSC-mechanoreceptor system in the rat and tested it against a variety of data drawn from the literature. The agreement was good enough to suggest that the model captures many of the key features of the peripheral whisker system in the rat

Emergence of visually-evoked reward expectation signals in dopamine neurons via the superior colliculus in V1 lesioned monkeys

Responses of midbrain dopamine (DA) neurons reflecting expected reward from sensory cues are critical for reward-based associative learning. However, critical pathways by which reward-related visual information is relayed to DA neurons remain unclear. To address this question, we investigated Pavlovian conditioning in macaque monkeys with unilateral primary visual cortex (V1) lesions (an animal model of 'blindsight'). Anticipatory licking responses to obtain juice drops were elicited in response to visual conditioned stimuli (CS) in the affected visual field. Subsequent pharmacological inactivation of the superior colliculus (SC) suppressed the anticipatory licking. Concurrent single unit recordings indicated that DA responses reflecting the reward expectation could be recorded in the absence of V1, and that these responses were also suppressed by SC inactivation. These results indicate that the subcortical visual circuit can relay reward-predicting visual information to DA neurons and integrity of the SC is necessary for visually-elicited classically conditioned responses after V1 lesion

A robot model of the basal ganglia: Behavior and intrinsic processing

The existence of multiple parallel loops connecting sensorimotor systems to the basal ganglia has given rise to proposals that these nuclei serve as a selection mechanism resolving competitions between the alternative actions available in a given context. A strong test of this hypothesis is to require a computational model of the basal ganglia to generate integrated selection sequences in an autonomous agent, we therefore describe a robot architecture into which such a model is embedded, and require it to control action selection in a robotic task inspired by animal observations. Our results demonstrate effective action selection by the embedded model under a wide range of sensory and motivational conditions. When confronted with multiple, high salience alternatives, the robot also exhibits forms of behavioral disintegration that show similarities to animal behavior in conflict situations. The model is shown to cast light on recent neurobiological findings concerning behavioral switching and sequencing

Segregated anatomical input to sub-regions of the rodent superior colliculus associated with approach and defense

The superior colliculus (SC) is responsible for sensorimotor transformations required to direct gaze toward or away from unexpected, biologically salient events. Significant changes in the external world are signaled to SC through primary multisensory afferents, spatially organized according to a retinotopic topography. For animals, where an unexpected event could indicate the presence of either predator or prey, early decisions to approach or avoid are particularly important. Rodents’ ecology dictates predators are most often detected initially as movements in upper visual field (mapped in medial SC), while appetitive stimuli are normally found in lower visual field (mapped in lateral SC). Our purpose was to exploit this functional segregation to reveal neural sites that can bias or modulate initial approach or avoidance responses. Small injections of Fluoro-Gold were made into medial or lateral sub-regions of intermediate and deep layers of SC (SCm/SCl). A remarkable segregation of input to these two functionally defined areas was found. (i) There were structures that projected only to SCm (e.g., specific cortical areas, lateral geniculate and suprageniculate thalamic nuclei, ventromedial and premammillary hypothalamic nuclei, and several brainstem areas) or SCl (e.g., primary somatosensory cortex representing upper body parts and vibrissae and parvicellular reticular nucleus in the brainstem). (ii) Other structures projected to both SCm and SCl but from topographically segregated populations of neurons (e.g., zona incerta and substantia nigra pars reticulata). (iii) There were a few brainstem areas in which retrogradely labeled neurons were spatially overlapping (e.g., pedunculopontine nucleus and locus coeruleus). These results indicate significantly more structures across the rat neuraxis are in a position to modulate defense responses evoked from SCm, and that neural mechanisms modulating SC-mediated defense or appetitive behavior are almost entirely segregated